Decision Trees and Attribute-Oriented Knowledge Discovery in Databases

We compare two types of knowledge representations for data mining: decision trees and knowledge rules, and three mining methods. We show that the decision tree representation and the knowledge rules representation are semantically equivalent. The ID3 algorithm by Quinlan has limited generalization power because it uses only a very simple generalization function of partial attribute removal type. ID3 may output too many knowledge rules. Quinlan's production rule generators use attribute removal functions that are more powerful than ID3. The HCC-algorithm by Han, Cai, and Cercone uses both attribute removal and concept tree ascension as its generalization functions. HCC-algorithm normally has more generalization power than the other two. HCC-algorithm also uses thresholds to control the output complexity. On computational eeciency, HCC-algorithm is the most eecient, ID3 is the next, while Quinlan's production rule generators are less eecient. We also point out some disadvantages of HCC-algorithm. It is possible to combine known generalization functions and strategies to form hybrid algorithms. For various applications, hybrid algorithms have advantages over the known methods. We propose a hybrid algorithm which combines all generalization functions and at the same time avoids over-generalization carefully. The proposed algorithm is stable, complete, and non-local and is about as eecient as Quinlan's production rule generators.